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This module is one of four developed to help vocational 
educators in state and local education agencies select, develop 
?or aaapt)! and effectively use vocational Competency measures. 
The precis module, Module 17, provided an overview of using 
competency measures in vocational education programs. The next 
module Module 19, presents a step-byr-step approach to developing 
the competency tests, and .Module 20 considers approaches to vac- 
ating competency tests and using test results. 

Competency testing emphasizes the practical aspects of a 
nob-can the student perform job-related tasks at the level 
expected by prospective employers? The purpose of this module 
is to help You arrive at the specific job-related competencies 
that need to be tes.ted. The discussion presented here is based 
on the experiences of the American Institutes for Research in .. 
conducting tne Vocational Competency Measures (VCM) project for 
the U.S. Department of Education. 



Overview 



Rpfore lumping into the development of a competency test, 
we're assuming that you've identified a need for testing a particu- 
lar program For example, maybe the electronics technician pro- 
grams in your district or state appear to be effective and there 
S l„ L * ereat potential for the future employment of gradu- 
HS 8 But the leve s of the graduates' skills and knowledge, 
acceding to local employers, seem to be uneven. Is there any 
way you can assure the employers that the students comp letxng 
your programs have achieved, a specific measured standard of per 
formance? 

We feel that vocational competency testing may be the answer 
and that this set of modules will help you resolve your dilemma. 
X r re of course, other reasons for using vocational compe- 

p T rog ms defined competency testing and f-usses the ways such 
ISifcri? can be used in vocational education programs -Competency 
tests are not new nor are they restricted to occupat ional tr aining. 
Most of us, for example, have taken the written and driving tests 
required to obtain drivers' licenses. Those tests measured whether 
we could perform certain tasks and had specific knowledge at a level 
hLh enough to meet predetermined standards. Using lists of safety 
sSndaTdSnd the laws regulating traffic, states have developed 
their competency measures for drivers. 



\ This module should assist you in setting standards and 
priorities— your list of competencies for job-related tasks. The 
following sections can also help you determine how adequate exist- 
ing measures are or provide the first steps, for developing, your . 
own measures. Then, if you decide to build a competency test, ■ 
you 'should be "ready to use the last two modules, Developing Voca- 
tional Competency Measures , and Validating Competency Tests and 
Using Test Results . > •' 

There is no question that the objective measurement of ^ 

occupational competency is becoming increasingly important in 6ur 
technological society. All four modules stress the development 
and use of competency measures to improve vocational programs for 
the benefit of future employees and their employers. 

Instructions to the Learner 

The Self-Check items and possible responses' to 'them are 
found in the Appendices. These questions have two purposes. 
First before you begin work on the module, you may use them to 
check'quickly whether you have already.learned the information 
in previous classes or readings. In some instances, with the con- 
sent of your instructor, you might decide to skip a whole module 
or parts of one. The second purpose of .the Self-Check is to help 
you review the content of modules you have studied in order to 
assess whether you have achieved the module's goals and objectives. 

You can also use the list of goals and objectives that follows 
to determine whether the module content is new to you and requires 
in-depth study, or whether the module can serve as a brief review 
before you^continue to the next module. 
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" Goals and Objectives 

Goal Is Perform the necessary steps for systematically determin- . 
ing requirements for vocational competency measures. , 

Objective 1.1- Involve expert advisors throughout the 
development of competency lists. 

Objective L.2 Collect and list job and task information for 
a specific occupation. 

Objective-!. 3 .Verify the task statements listed on a compre- 
hensive task inventory. 

J * . 

Goal 2: On the basis of ■ identified task priorities, list specific 
7oriplt"encies for reviewing and developing competency tests. 

Objective 2.1 Analyze the data to determine priorities 
among tasks. • 

Objective 2.2 Revise the' lis* of task statements and develop 
an outline of specific competencies. 

Objective 2.3 Use the list of competencies- to review exist- 
ing vocational' competency measures. 



Resources 



In order to complete the learning activities in this module, 
you will need information contained in the following publication: 

Erickson, R. C. , & Wentling, T. L. Measuring student 
gro wth: Techniques and procedures for occupati onal 
education. Urbana, 111.: Griffon Press, 1976.' 
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GOAL 1: Perform the necessary steps for systematically determin- 
ing requirements for vocational competency measures. 



How Do You Determine. What To Measure? 

This module Ascribes a step-by-step process for determining 
requirements for National competency measures. The outcome of 
this process should be a preliminary set of competencies for a 
specific occupation. The list of competencies can then be used 
to help you decide whether to (1) use an existing test as it stands, 
(2) modify an existing test, or (3) develop your own test. 

, L 

Enlist the Aid of Your Strongest Allies 

A range of individuals should be involved from the earliest 
decisions about competency testing to the final rounds of updating 
tests and using the results for program improvement. Employers are 
.Tpe'ciSly important, of- course, because they hire the people . you 
train But, equally important to test development and use, is the 
foyers' help in job and task analysis-the basic building block 
ror developing and evaluating vocational educatiorf programs. 

' Tn developing the AIR Vocational Competency Measures, we asked 
for help nationwide from a variety of groups with an interest m voca 
Monal education and a concern for preparing students- for ^cessful 
entry into jobs Individuals from these groups became members of our 
advisory panels, resources for locating existing competency measures 
and task analysis lists, and reviewers and authors of test items. 

Most likely -you will draw upon representatives of the same 
agencies and organizations # in your state or region, including the 
following: % / 

• Vocational teachers and administrators (secondary, 
postsecondary, and proprietary schools) 

• Business and industry (both small and. large employers) 

• Unions and apprenticeship representatives 

• State and' local advisory committees for vocational 
training programs 
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• State agency* staff and occupational, 'specialists 

• Professional association leaders * t 

Your advisory committee should be a working committee-,, indi- 
viduals who are committed to improving vocational *educat'ioh, 
knowledgeable about the selected occupational areas, and willing 
to work the necessary hours for^ several months. It is especially' 
important that the committee include' individuals who provide in- * 
sights into the special needs .of minorities", limited-English 
speaking groups, women, *and the handicapped. 

Working with advisory committees .. There is no magic formula 
for working efficiently with advisory groups. Someone har> to be 
in charge, starting with selecting the committee and then making 
sure that the following requirements are carried out: 



/ 



regular nteeting times 



• organized agendas 

• -clearly spelled-oii? responsibilities — group and individual 

\ > 

• time limits on reviewing materials' and information 

• • open and consistent communications by mail and phone 

regarding deadline^ accomplishments , problem's" - 

• appropriate recognition for each individual's efforts' 

Once you've selected an advisory committer to help .you on the 
way to competency testing, the next step is to analyse the specific 
job -to be tested. The key question to keep in mind throughout the 
riext step is: * 



What's important for vocational program^ graduates to 
know and be able to do before they enter a specific 
job market? 



Collect Job and -Task Information 

What kinds of job information exist and how can they be used? 
Job information is useful in helping you identify job components 
and document, the duties and tasks performed by employees. The ques 
tions' asked in analyzing any* job are: 
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• ' What^does the worker do? 

• How .does the worker do it? 

• Why does °the worker do it? 

• # • 

• What skills 'does the worker require to do it? 



Most likely! you will n<k be conducting a full-scale job 
analysis- ratheJ you will be deriving a list of duties. a*d tasks 
f^om Panoses already done by others. A job analysxs usually 
includes the following: • . • 

1. A general description of the work to be done;; 

,2. The organizational setting and relationships in which ^ 

\ • the work is accomplsihed ; . , 1 

• \ '' • • 

3. ' 'The specific tasks that are to be carried out by the . 

worker; • * . ; 

4. The equipment, tools, ,and, materials to be used; . ^ 

5. The working conditions, including special hazards ; 

6 The qualifications required to learn and to perform 
• these tasks stated in terms of knowledge^ skills, abilf _ 

ties, and personal characteristics; <^ 1 , 

• 7. The expected manner ot quality of performance; ♦ . 

8. The process and expected outcomes of the performance. 1 • 

The work of -breaking down" an ^existing job analysis ^ ~ e " . 
overwhelming as it may seem at first glance To si mplif 
ess the primary $ocus should be on the- third item- the specific 
tasks! however; a]W the above elements will be useful. , . 

The employer members of your advisory committee areprobably 
the best source of information about the tasks that make up the 
selected ob Other sources that can be tapped for job, and task 
information include the Department of Labor, trade and professional 
-associations" and the Department of Defense. QSee the Appendices , 
for a listing of references and information sources.; 

Before going any further, we should, define what we mean by a 
task imply stated! a task is a distinct action that is observable 
and Measurable, with a specific purpose and a clear beginning and 



-11- 



12 



ending. Figure 1 below illustrates the relationship between duties 
(broad descriptors) and tasks (specific descriptors). Constructing 
a task inventory is much easier if you use major duties as headings, 
and then group related task statements under those headings. 



JOB 



Restaurant Services 
(Waiter, Waitress, Cashier) 



DUTIES: 



Dining Room 
Service 



Cashiering 



Sid^ework 



TASKS: 
(Partial 
Listing) 



Greet 

and 
, Seat 
Customers 



Take 
Customer 
Orflers 



Operate 

Cash 
Register 



Process 
^Credit 
"Card 
§aJLes 



Figure 1. Example of relationship between job, 
duties, and tasks 



Develop a Task Inventory 

The easiest way to develop a comprehensive task inventory 
is to use what's available and build from that point. After you've 
amassed everything you can find about a job, then you and your 
committee need to consolidate the information into a task inventory. 
This inventory should become a comprehensive listing of basic task 
statements that will be reviewed, revise 7 and refined into a set 
of task priorities for the selected job. 
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Fi U in the gap. on your task inventory using j-™^*" 
curricelum guides and fro. your a isory -Jj-J^ M f u . 
this point, U s better to hav o( this preliBi „ary task 

^V^l^^ irrelevant items than to cohere up 
tasks' that should be included. _ 



^ best ^^e7^uV^T^ sZ 
%ZX?££X accurately reflect current 
job requirements. 



Writin^ask^tatem^. fT^^S^*^ ' 
statements listed as a series of dl struct acti i rf 

-fflS'^S^ fn b pro P :r sequence in order 
£r the job to be completed satisfactorily. 

0nc e you've consolidated the task statements fro. your review 
of existing task analyses, revise the "«™ents s . atenents 
the levels of effort are comparable Of course a 
mus t be written so that they are clea J-J^J^ st ^ eBents 

SEafare^ vatu"! £ ^"at 'are too detailed.* On a list 
3"LE fo°r a ustom sewing Us, : for Jj."-.^^; The 

Exam ples of Task Statements 

. „ u Obiect gualifging Phrase 
Action Verb U PJJSS. 

' the water using the appropri- 

Measures samp le ' ate test instrument 

by phoning to obtain- 
Verifl6S payments credit information 



Ob.jec 
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Keep the following points in mind during your revision: 

• A task statement begins with an action verb to indicate 
what a person does on the job. This is usually an 
observed behavior. 

• The action verb is followed by an object — the thing 
or person receiving the action. 

• The object may be followed by a qualifier — to make 
the task more specific, describe the expected outcome, 
or to detail the tools or equipment used to carry out 
the task. 



Verify Task Importance ' 

Verify the importance of each task by surveying employers. The 
task inventory you've compiled can be used as a job-specific question- 
naire written in the "language" of the job. As shown in the follow- 
ing partial task inventory for Restaurant Service, related tasks are 
grouped under functional headings that represent duties. Similar 
questionnaires for each of the tests we developed were administered 
to employers and employees nationwide. 

Ask your advisory committee to nominate local and state businesses, 
naming supervisors they feel would cooperate in completing task 
inventory questionnaires. It is best to work with a range of employ- 
ers, particularly middle-size businesses. We found that, in general, 
middle-size companies were often good candidates for cooperative sur- 
veys because they appreciate the value of such work, but do not have 
the resources for conducting the analyses. 

In addition to obtaining employer ratings on the frequency that 
each task is performed and the relative importance of each task, ask' 
employees working- in the specific job area for the same" information. 
Both groups should respond to the question about whether each task 
should be Learned before employment (i.e., in a training program) or 
on the job. The last page of the task inventory asks respondents for 
the personal qualifications they think are important in order to do 
the job, and to rank the 10 tasks most critical to the job. 

Other methods for gathering and verifying task information, As 
shown in Table 1, there are several ways to check whether the tasks 
listed in your task inventory are accurate, complete, in proper 
sequence, and easy to understand. The choice of methods depends on- 
the available amount of time and money and the nuipber of people you 
have to carry out the verification process. Sometimes i"he careful 
review of a task inventory list by a small group of experts represent- 
ing the range of employers that typically hire your graduates can 
yield as much good advice as a widespread, expensive survey. 

-14- 
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TASK INVENTORY* FOR RESrAURANT SERVICE (WAITER, WAITRESS, CASHIER) 

Ha,ik £i ound I n f < irm/i U on 



Job 111 le: , Employ*.- i 



Miches I education grade complete*! : 

Spec In I courses completed (or present work 
(include apprenticeship or military courses) 



A) . o: Sex: Years of experience hi present work: 



Task List 

Instr uct hws \ i i . 

t".,; folium,, list of tasks n,ny ho ^rformedj.v who work In ™»«.t service <w»Uer ^^•^ *' .'S" 

Wo would like lo know which tasks .ire perform) on this job, how often they arc performed, how Important tncy 
Job. and when they should be learned (before or after entry to the Job). 

1. How often - Is the task performed on this Job? 

' NOP ,H»NK If the task is not required. (NO.K: II the task Is not reared, do not answer other , M ,e S . Ions nboot the task.) 
jSHTtTTf the task Is performed once a week, dally, or more often. 
MONTnT.Y If the task Is performed about once a month. 

(JUARTKRLY If the task Is performed about every three months oi less often. 

2. How Important to the overall |ob Is each task* 

U10H If the task has hjj>h Importance. Is a., essmijtjal |»nrl of the |oh, and definitely Influence-: effectiveness of 

lob performance. t . . 

MODERATE If the task has mouVralc Importance, contributes to the Job. but the job doe.su t depend on It. 
LOW If the task Is of Importance and does not contribute much to the job. 

*j. hTthe worker leqiilrcd to perform the task upon entry to the |oh or Is It learned on the Job? 
BEFO RE If most of the task should be learned betoic entry to the Job. 
AFTER "if most of the task should be learned after entry to the Job. 

Frequency Importance Should Be Learned 

CENERAL TASKS i NOT wt ? FKl Y MON'IMI Y gUAH " 1 1 1 ( Ml ' MODERATE LOW BEFORE AFTER (Entry to Job) 

. |)OME " " rERLY 

room appearance 

I. Handle customer romp I a In ts 

J. Report equipment needing 
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Frequency 



N0T WKRKI.Y MONTHLY 

DONE 1 fcKM 



Importance _ Should 

11,011 MODRKATE LOW BKFORR AFIKR (Entry to Job) 



9. Make reservations/ 
waiting lists 

10. Gieet and seat guests 

IK Provide menus and answer 
Inquiries -about menu 
Items 



12,. 



Suggest menu Items 



n. Take guests 1 orders 
and turn In orders* 

14. Load and enrry i rays 

15. Serve courses In 
correct sequence. 



CASlHRKlNU t 

50. Prepare cash drawers 

51. Accept payments ^ 

'52. Opeiate cash register 

53. Write and add checks 

(tabs)" 

04. Process credit card sales - 

*>5. Authorize customers 1 ^ 

check* and process 

56. Compute tax - 

57. Verifv i ash in dr.iwer 
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AlltllTIONAL INFORMATION 

A . mm S « eorgogol Qualities that are ..portant l« order'to do t M . |ob? (Kxa^: Beh,« 



T 6 - 



N.im<. work liabUa (Example: Srarj work on time) 



N..m,. t.»tei-|»0WOH«l relations (Exn-gle: Accept supervisor's surest Ions) 



B. Could Important parts of this |ol. bo done by persons who are": 

Physically l,.»o«l capped? UUu\ or .visually Impaired? 



Deaf or hcarlnfi in.pal.ed:? Mentally retarded? 



Do you 



k „ow or have vou heard of a person with so,„e kind o^ handicap who successful perfo-.ns this |*. 
No m 

Defii-rlbe the handicap mid .mv modi f lent Ions to the H>. 
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TABLE 1 

METHODS rOK GATHER I!IG \ND VERITY 1SG JOE AND TASK INFO3IIATI0K 



ItctllOtLs 


Procedures 


Advantages 


Disadvantages 


Interview lug 
Workers and , 
Supervisors 


Use n general interview gul«V or a guIdV 
developed fur a specific job. 

Call or visit empt overs/employees lu 
ici/ uiiif Mm omn 1 nvpp dne*i . what mnctilncs 
and materials are used, which tasks aie 
must critical and how often they are done. 

6 * 


Easy to probe for answers, 
particularly regarding atti-' 
tudt»s, and personal quali- 
t I rat ions. 

Unclear or ambiguous respon- 
ses can be clarified^. 


N 

Time consuming and 
expcusl vesto obtain 
large number of res- 
pondents. 


Using Que*- 
C lonunl res 
( 1 uc lulling 
Task 

1 nvonlorv 
l.(sts) 


Develop a questionnaire at ouiiiL-tho com- 
prehensive list of tasks compiled to make 
up a job description. Use scales for 
f requeue v nf task performance and Impor- 
tance of each task, hake 3uro questions 
are brief and easy to read and understand. 
Leave room for additional comments frnm 
respondent*. B'» sure questionnaire format 
.i I lows for easy data analysis. 


Least expensive way to 
obtain first-hand informa- 
tion from actual Jnb sites, 
Including employers and 
employees . 

Lfflclent methnd for gather- 
ing large amount of- jnb 
hifoin-allon . 


Uses employer/ 
employee time for 
eomplet Ion. 

May require follow- 
up by m j I 1 or phone 
for complete In for* 
mat Ion. 

Requires time for 
analyzing data. 


*» 

Observing 
Job Per- 
formance On 
Site 


Develop a checklist before the actual 
observations, listing the general task 
categories expected to be observed. 

Observe the entire work cvHe on site, 
listing llie Job behaviors, equipment used, 
and sequence of tasks. 


Host method for clarifying 
urn* 1 ca r procedures for 
carrying out a specific Job. 

Out-of-date documentation of 
tasks ran be updated on 
site. 


Time consuming anil fc 
relatively expensive, 
depending on tho 
number of sites 
visited. 


Reviewing 
Documents 
(i nc hiding 
logs or 
journals of 
tvplral work 
davs) 


Review 1 I terat ure/doenments describing a 
specific Job, and consolidate the Infor- 
mation In logical sequence under ma|el 
dutv headings. Use Information to 
develop task Inventories, interview 
guides, and observation checklists. 


Good wav to fill in gaps 
from observat Inns on site, 
and to prepare preliminary 
task inventory lists. 

Least expensive method. 


Information wav he 
outdated or lacking 
fnr new and emerging 
occupat Ions. 
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Individual Study Activities 

1 Select an occupation, with vhich you are familiar preferably 
one in Xich you have worked. Write a brie rf Jol > £"«iptxon 
hv listing the general duties, most important tasks, and 
specJa worSn g S conditions, tools, etc for carrying out 
the tasks. Refer to the Recommended References in this 
module or one of your own resources for information on 

ng Jo descriptions. Then compare your descript ion 
wirn the job description in the Dictionary of Occupational 
Titles (DOT) for the occupation you selected. 



2. 



Read at least one chapter of a text from the Recommended 
Reference list that pertains to task inventories-- (a) their 
development, use, or verification; (b) the relationship to 
job analysis or competency measurement; or (c) their use in 
curriculum development. Summarize your reading and share 
the information with the class. The various readings by 
class members should allow the identification of at least 
four different approaches to organizing and conducting task 
analyses. 



Disc ussion Questions 
1. 



Using community employers effectively as advisors in helping 
you develop competency measures requires tactful ^adership 
and efficient management. Discuss the advantages. and some 
of the problems that can arise when working with employers 
Sow would you, as vocational educators, go about selecting 
and managing an advisory group for developing a competency 
test? 

2 Discuss the advantages and disadvantages, of "using existing 
-ob and task analyses for developing a particular competency 
Jest for use in your district or state. Consider whether^ 
factors such as regional differences, state licensing, trade 
•practices, mandated curricula, etc., could affect the use of 
any existing task analysis. What modifications might need 
to be made to adapt task statements to your circumstances?^ 
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Group 'Activities 

1. Depending on the size of the class, form groups to roleplay 
an advisory committee. Each group member should assume a 
role that represents the categories listed on pages 9 and 10 
under Goal 1. The groups should then select a chairperson, 
set an agenda for the meeting, assign responsibilities and 
due dates for activities, etc. The main purpose of each 
group should be to' establish a. working relationship with 

the goal of selecting a single occupation for which compe- 
tency, measures will be developed.* 

2. Divide the class into four groups, with each group selecting 
a method presented in Table 1 on page 18 for gathering and 
verifying inf prmatioh. Either use an existing task inven- 
tory or develop a task inventory for a specific occupation . 

* to carry out the procedures described in Table 1. Whenever 
possible, verify the tasks with actual employers in the 
community. 
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GOAL 2- On the basis of identified task priorities, list specific 
' competencies for reviewing and developing competency tests, 



• 



What Do You Do With All the I nformation? 

At this stage, you most likely have gathered job information 
from a variety of sources and used several methods to collect data. 
To summarize the steps .we used to arrive at this point: 

Select and meet regularly with your advisory committee. 

Collect all available task analysis information for the 
selected occupation. 

• Consolidate existing information into a comprehensive 
task inventory. 

• Verify the task statements' with employers and workers 
by questionnaires, interviews, or observations on the 
j ob. 

• Collect information about task importance and frequency 
of performance for use ih selecting, tasks to include in 
the tests. Information on task difficulty,- critical 
factors in task performance, and criteria for judging 
adequacy of performance will provide useful background 
information for item development.. _ , 

The checklist on the following page will give you a quick look 
at what you've accomplished and at' the steps that are still to come. 
The next step is to analyze the information to identify the tasks 
that will be used in developing a competency test. 



Analyze the Data 

-The actual analysis of data about the frequency, and the impor-' 
tance of each task performed can be a simple tabulation qr-.can - 
involve complex statistics. We suggest the former. Remember, that 
your final list of competencies should include all tasks that are 
judged by most supervisors and their employees to be (1) important 
to the overall job, and (2) performed frequently by most entry- 
level" workers , those -beginning employees who have already received 
some training for a specific occupation.. 
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CHECKLIST FOR DETERMINING COMPETENCIES TO 3E MEASURED 




Develop Initial Task Inventory 
(Obtain recommendations, < 
existing lists' from) * 



Advisor/ committee 



Employers 



Skilled workers 



Related job and cask inven- 



2. Develop Task Inventory Survey 
Questionnaire 



Group related tasks unaer 
duties 



Develop scale for rating 
Importance of tasks 



Develop scale cor racing fre- 
cuencv of cask performance 



Requesc racings on cricical 
ga3ks 



3. -Conduce Task Invencorv Survey 



Survey supervisors and cheir v 
employees using quescionnaire 



Conduce surveys of a small 
3ample of similar employers 



Analvzc Survey Data 



Average frequency daca across 
all supervisors and across 
all employees 



Average imporcance racings 
across supervisors and 
across employees 



Weight supervisor responses 
and obtain overall racing for 
gach task ; 



5. Decermine Task Priorities 



Delete casks raced not done 
or uninporcar.c 



Group remaining casks under 
ouciine of major ducy head- 
ings 



Ensure chac casks judged as 
cricical are included 



Obcain experc review of re- 
vised list cf .comoecencies 



Reviev Existing Tests 



Check cities of available 
competency measures (FL, 
NOCTI, Buros, ecc.) 



Compare list of cask pr-i- 
oricies vich available 
cescs; noting gaps, compe- 
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Let's .assume that you have sent task inventories to several 
hotel managers with the request that they rate the ^quency 
importance of each task on the list. The following are parts of 
a Tall J inventory questionnaire Returned to you by a supervisor of 
hotel front-desk employees. Ths supervisor has circled the 
critical tasks and checked the appropriate columns. 



TASKS 



FREQUENCY 



IMPORTANCE 



General Tasks 



Not 
Done 



'Daily or 
Weekly 



Monthly 



Seldom or 
Quarterly 



High 



Moder- 
ate 



Low 



Q Greet guests 

2 Arrange tours 
.for guests 

3 Schedule 
shifts for 
desk clerks 



>/(3) 



✓ (1) 



v/(0) 



/(OX 



= 7 
•1 



Processing 



Guests 



35 Verify 
guest 
reserva- 
tions 

36 Arrange 
payments 
from guests 



7) Check out 
guests 



v/(4) 



= 7 



8) Follow up 
unpaid 
bills 



y/(2) 



H2) 



Your first step would be to assign a numerical rating to each 
column. As shown above, we assigned "0" for "Not Done"; "3" for 
"Daily /Weekly"; "2" for "Monthly"; "1" for "Seldom/Quarterly. Then, 
the checks for "Importance" were weighted to ensure that essential 
tasks would not be deleted if they were performed less often,' That 
is, checks in the "High" column received a "4"; "Moderate, a 2 ; 
and "Low " a "0." The numbers were summed across each item and 
each total was recorded. As you can .see, "7" was the maximum total 
any item could receive. We -used the same procedure and scales for 
the task inventory questionnaires completed by employees on the job. 
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As stated earlier., supervisors and employees were also asked 
to circle the 10 tasks on th& inventory they felt were most criti« 
cal to job performance. Judgments of criticality provide a go'od 
way to verify task importance ratings. For example, we found* 
that about 60% of the task's on all our inventories that received 
a total rating of "5" or higher were also judged as "critical 11 by 
both supervisors and their employees. 

Next, we summarized all the ratings frpm all respqndents.on 
a sheet similar to the following example: t, 



TASKS 



Salience, Salience, Critical Overall Rating 

General Tasks Supervisors Employees (at 'least (Supervisors 

(Average) (Average) one mention) and Employees) 

1 Greet guests 7.0 „ 7.0 11 7.0 

2 Arrange tours 2.8 0 - 1.9 
for guests 

3 Schedule shifts 2.3 1.0 - "1.9 
t'or desk clerks ♦ 



Processing Guests 

35 Verify guest 6.0' 
reservations 

36 Arrange payments 7.0 
from guests 

37 Check out guests 6.3 

38 Follow up unpaid 4.0 
bills 



6.6 . 8 , 6.2 

7.0 12 7.0 

6.4 11 

1.0 - 3.0 



Averages were computed for "the total** dumber of supervisors and' 
for the total number of employees responding. to each item. We , 
wanted to give greater weight to the ratings of supervisors and 
still include tasks rated very high by employees but only moderately 
high by supervisors. Therefore, we used the following formula to 
arrive at an' overall rating for each of the tasks: 

_ 2S + E 
3 

R » overall rating * 
S s supervisors 1, average 
E » employees 1 average 

For example, supervisors gave an average response of 6.0 to Item 35, 
Verify Gu\^st Reservations; ;on the same 7-point scale, employees 

- $ 
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rated it 6.6: Using- the formula, we would have 

n 2(6) + 6.6 _ U+_6^6 6 2 

• »v — 5 3 3 

When thisMs-done'for each* item on the inventory questionnaire*," 
the result is^a lisf of tasks and their weighted importance. 



* In JUd'ition to determining the importance, frequency, and 
* criticality of each item -on .the comprehensive task -inventory, it. 
■ would' be helpful to check whether each task should be learned 
before or after entry on the job. .Afeain, we suggest giving the 
ereatest weight to the supervisors' responses. For example, it 
< SUrhirds or more .-of- the. superv^ors felt that diesel mechanics 
'sWld J.earn to check turbochargers-' on the Job 'and not in school,- < 
we woull question whether fhat task should -be included -on a per- 
formance test 'for students. Questions'of this type are probably . • 
best solved by asking .advisory committee members or consultants 
experienced in the spetific occupation. - - _ 

Finally, the las* pag? of 'our task inventories included ' 
questions about .personal characteristics, such as desirable work % 
habits, 'and whether handicapped* persons cou}ld successfully, per- _ 
form the particular job. Some of the responses were especially 
helpful in pointing out to the test developers, some attitudes 
and behaviors that should be tested. -Although we fejt ^J^- 
verse had been covered in our task. inventories, blanks were left 
at the end of each Section of tHe'-questionnaire to encourage 
respondents ? to add any overlooked' tasfcs.. Most Additions however, 
were found to be specific to a. respondent's particular job situa- 
tioru - o • \ 

" ■' A word 9f caution! This is not the only, way to conduct a 
survey of tasks and analyze data. Any method that seems reasonable 
for your situa iqn and that, is coVsisifent ,. clear and methodical 
Should worku We present this information as a^guide that you are 
encouraged to_adapt or modify. , ( \. ' - , 

Determine Task Priorities * 

Everything you've. done on task analysis has been- accomplished ■ 
fo- the purpose of determining task priorities. Which tasks .a^e ^ 
most important to getting the job done? .The answer to that ques- 
tion" becomes the list of task priorities- on which existing compe- 
tency measures will be reviewed "or new measures will be developed. 
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In our analysis* of hundreds of responses to the various task 
inven ory lis ts .'"we discovered that certain tasks were rated con- 
sistently as critical and were performed regularly and often 
^tieS ta sks^had^ovTr niTItEgTorsarw -h±gher-(as-det-er«-lned- - 
3 the Tafiency formula described previously)'. Such tasks should 
'be ncluded in'the final U,t of competencies. . Conversely other 
"tasks clearly fell along the wayside,- with overall ratings of 3.0 
' or less These were then deleted/ For task! that seemed mar- 
glnal - around 4.0), we suggest you look closely at the super- 

visors' overall rating or check with knowledgeable advisors, 
'.^metimes, "borderline" tasks appeared. redundant--tasks had 

ready been included to cover that particular skill adequately-- 
and they were dropped, i'When iri doubt", do not leave it out may 
be a wise "rule, however , ..especially if,-there'i; some concern 
about Wing' enough" tasks (about 50) for test development. , 

< : Finally', rebuild 'the list .of surviving tasks by: 

' . logically grouping' separate tasks~-under major headings 
of duties; ;..*'" * 

. . . ' ^viewing wording of tasi^statements for clarity^and 
■consistency; and Np 

. ' • insuring that tasks are comparable as to level of 
» * de*ail' % (may Jhean combining task statements) _ ■ 

. At this. stageVfou may want to ask your advisory pAel 
to review criti*ally/the^evised list of task statements* An 
' expert review' should t&u^-in .an 1 -outline of competencies that is 
comprehensive and prov?des y a useful tool for the next «tep. 
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Review Existing Tests \\f/ ^ • 

* You'ue now arrived at a poifit- where you know which set of 
tasks y you'Ube using to eithe'^ review; ex^ing tests or develop 
4oor4wn measures. Many different' competent tests "have been 
develop^ "over the last several yea^. Yo,u may not have to develop 
youf^if you carefully screen available measures and fin* they 
Let your^eds-. Currently, there are 49 occupational competency 
tests nearing completion-.. The tests- and test developers are listed 
by occupational area in Table 2 on, the .following: ^ges . In. addi- 
' tion to those te^t-s, the Appendices section provides references to 
other efforts in Vocational competency measurement. Standard 

referencesjsfituld also be -looked at, such as • the Mental 
SSLnrement Yearbook * (Buros , 1978) and others included in the 
Recommended -References on page 45. _ t , 

* 
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TABLE 2 

VOCATIONAL COMPETENCY TESTS NEARING COMPLETION 



, -AREA/TEST — . — — — ~ 

AGRICULTURE ; 

Agricultural Chemicals Applications 
Technician 

Agricultural Mechanics 
- Fara Ea.uipnent Mechanic 
Greenhouse Grower 
Horticulture 
Nursery Worker 
Tractor Mechanic 

BUSINESS AND OFFICE : 

Accounting/Bookkeeping* - 

Clerk Typist 
Ccnputer Operator 
General Office 

Secretary (includes Stenographer aodule) 
Word Processing Specialist 

DISTRIBUTIVE EDUCATION : 
Apparel Sales 
Fabric Sales 
General Merchandising 
Grocery Clerk 

Hotel (Motel) Front Office 
Salesperson, Parts 

HEALTH: 

Dental Assistant 
Hospital Ward Clerk 
Nursing Aide 

Physical Therapist Assistant 
Practical Nursing Occupations 



AIR J? Ek 



(continued) 



*AIR - The American Institutes for Research Vocational Competency 
H - MCTlTtucent'occupational Competency Achievement Testing 

Program _ 
FL - Florida Occupational Achievenent Tests 

(See Appendices for addresses.) 



28 



TABLE 2 (continued)' 



AREA/TEST 


AIR 


N 


FL 


HOME ECONOMICS: 








Child Care Worker 






X 


Comaercial Cook 






X 


Custom Sewing 


X 






Fashion Construction Services 




X 




Restaurant Service' (Waiter, Waitress, 








Cashier) 


X 






TECHNICAL: 








Electronics Technician 


v 
A 






Wastewater Treatment Technician 


X 






Water Treatment Technician 


X 






TRADE AND INDUSTRY: 








Auto Body Repair 




X 




Auto Mechanics 




X 




Bricklayer " 


- 


X 




Bricklayer (Construction) 






X 


Carpenter 


X 






Construction Electricity (House) 




X 




Diesel Mechanic 


X 






Drafting 




X 




Heating and Air Conditioning 




X 




Industrial Electronics 




X 




Machine Trades 




X 




Plumbing 




X 




Printing 




X 




Radio/Television Repair 




X 




Refrigeration 




X 




Small Engine Repair 




X 




Welding 




X 





2 

In reviewing available competency measures, you will naturally 
look at comprehensiveness — does the test cover all the duties and 
tasks you've identified as important? What areas kre inadequately 
measured, or what duties are superfluous to entry-level testing 
needs? For example, you may have unearthed a retail clerk test 
that meets your competency requirements adequately except that it 
slights customer relations. You might consider using that test, 
but, in addition, developing a customer relations test to use as • 
a companion measure. 
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Reviewing existing measures also means paying close attention 
to test bias. Tests must be fair, sensitive to, and usable with 
special populations-minorities, limited-English proficiency, 
women, the handicapped. Carefully study the technical manual for 
the test to determine whether the test has been tried out with 
special pdpulations, and what evidence is available oft test 
reliability and validity. 

Then, if all else' fails, you may be faced with the creative 
and rather complex adventure of developing your own competency 
measures. You might first consider asking your advisory panel 
to corroborate your decision to develop a test; then enlist their 
aid in the development process. They can help you by suggesting 
' performance measures for specific tasks, and by recommending 
individuals who could develop test items for a paper-and-pencil 
test!. This information is necessary for the work described in 
the third module of this series, Developing Vocational Competency 
Measures. 
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Individual Study Activities 



1. If possible, obtain an example of an occupational competency 
test (check V-TECS Progress and Information Reports, NOCTI 
Annual Reports, Buros, and other general test reference . 
books) . Review the test as if you were deciding whether to 
use it in your district. Briefly note where you feel gaps, 
redundancy, unclear or biased- wording, etc., exist. Pre- 
pare a short critique of the .selected test for a presenta- 
tion to the class. 

2 With a partner, select one occupation from the DOT. Then, 
individually, develop a series of task statements for the 
selecte'd job.' With Tour partner, 'compare, your task state- 
ments, looking, carefully at the discrepancies. -Finally, 
combine the two, rearranging and grouping the tasks logically 
under major duties and clarifying the task statements. 

Discussion Questions 

1 A major problem in developing competency-based vocational 
curriculum and achievement measures is reaching . agreement on 
what competencies ought to be taught and. how those competen- 
cies can be measured. Consider your district or state 
requirements and local union standards for a selected occu- 
pational program in a discussion . of selecting competencies 
to be measured. 

2 For the same occupation (s) considered above (or others)', 
discuss the constraints or flexibility necessary in develop- 
ing competency measures for the following special student/ 
employee groups: 



disadvantaged 

• limited-English proficiency 

• handicapped 

• displaced homeraakers 
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Group Activities " ' 

1. For this activity, individual class members are responsible 
for inviting employers' and educators to speak to the class on 
their needs fpr easy-to-administer , comprehensive, unbiased 
competency tests. Recruit a panel from the following cate- 
gories of resource people: local director or. dean' of voca- 
tional education; district or area curriculum coordinator; 
supervisors and personnel staff from local or area employers; 
occupational specialist from 'district or state; trade union 
representatives; and so on. Formulate questions prior to 
the panel presentations on topics such as: who and how to 

"decide whether to use competency measures; how to determine 
the competencies that' should be tested;, relating actions for 
competency measures to school programs versus employer 
demands; improving occupational programs; and using compe- 
tency tests to screen potential employees. 

<• 

2. Assume that your district plans to develop a "competency test 
for an occupation in an expanding job market in your state.. 
Select the occupation for which you will outline an action 
plan listing the duties and responsibilities that should be 
covered to develop the test. Divide into small groups, with, 
each group drafting one section of the outline: 

a. Selecting and managing an advisory committee 

b. Developing a task inventory 

c. Verifying task importance 

d. Determining task priorities 

(Depending on class size and the time available for this 
activity, the groups can tackle smaller sections of work 
as divided' in this module.) Although you most likely will m 
not be able to do a full-scale analysis of desired competen- 
cies, your end-product as a class should be a'workable, 
practical guide to fit your district, school, or state needs. 
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Summary 



This module has described systematic and logical procedures 
for developing a list of competencies for use in developing or 
reviewing vocational competency measures. Whatever the r back- 
ground and experience in formal testing, most vocational educa- 
tors wm find themselves increasingly involved .in competency 
testing." Therefore,- it is important f o know that the steps 
neeaed S to select competencies are similar for both reviewing 
existing tests and for developing new. measures. Those steps 



are: 



• Establish a working advisory group. 

• Develop a comprehensive task inventory. 

• Verify the tasks by surveying employers. 

. Determine task priorities by analyzing survey data. 

• Outline revised task statements for use in 
reviewing or developing competency measures. 
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Sources of Infnrmation abou* Tnh and Task Analysis ■ 

* 

. National Center for Research in Vocational Education 
The Ohio State University 
1960 Kenny Road 
Columbus, OH 43210 

• Centers of the National Network for 'Curriculum Coordination 
in Vocational Education. (NNCCVTE) : . 

William Daniels, 1982 NNCCVTE Chairperson and Director 
Northwest Curriculum Coordination Center 
.Building 17, LS-10 
Airdus trial Park 
Olympia, WA 98504 
206/753-0879 

Rebecca S. Douglass, Director 

East Central Curriculum Coordination Center . 

Sangamon State University, E-22 

Springfield, IL 62708 

217/786-6375 

Lawrence F. Ht. Zane, Director 
Western Curriculum Coordination Center 
.University- of Hawaii 
. 1776 University Avenue 
Wist Hall 21.6 
Honolulu, HI 96822 
808/948-7834 

Roy Hinrichs, Director 
Southeast Curriculum Coordination Center 
Mississippi State University 
P.O. Drawer DX 
Mississippi State, MS 39762 
^1/325-2510 ' 

Bob N ?atton, Director 

Midwest Curriculum Coordination Center 
.Curriculum & Instructional Materials Center 
1515 West\6th" Avenue 
Stillwater\0K 74074 . 
405/377-2000. 

Joseph Kelly, Director , 
Northeast Curriculum Coordination Center 
Bureau of Occupational & Career Research Development 
Division of Vocational Education 
225 West State Street\ 
Trenton, NJ 08625 V 
609/292-5850 
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American Association Community and Junior Colleges 
1 Dupont Circle, N.W.£^ 
Washington, DC 20036. 

U.S. Department of Labor 

U.S. Department of Defense 

Mid-America Vocational Curriculum Consortium 
1515 West Sixth Avenue 
Stillwater, .OK 74074 

Vocational-Technical Education Consortium (V-TECS) 
Southern Assoc. of Colleges & Schools 
795 Peachtree Street, N.E. 
Atlanta, GA 30308 



State Agencies and Consortia 
Developing Occupational Competency Measures 

Florida Department of Education 

Bureau of Vocational Research,. „ / 

Dissemination and Evaluation 
Knott Building 
Tallahassee, FL 32301 

National Occupational Competency Testing Institute 
A5 Colven Avenue 
Albany, NY 12206 
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Self-Check 



GOAL 1 

1. What is the foundation of a good vocational competency 
test? 

2. How can advisory groups help' in the development of compe- 
tency measures? 

3. Describe the relationship between job, duties, and tasks. 

4. What parts make up a clear, concise task statement? 
Give one example of a task statement. 

5. State four methods for collecting and verifying task 
information 



GOAL 2. 

l^When selecting tasks to be covered in a vocational competency 
TeTt, what are the two most important pieces of information 
to analyze?- 

? What other types of information should be compiled during 
Tail verification' in order to provide" background information 
for item development? 

3 When you have selected thofee tasks' that should be ' * n ' lud **f 
a vocational competency telt, what two steps should be accom- 
plished befpre you start to develop the test. 

4.- Tests of all types should be carefully screened for bias 
against which four vocational education subpopulations? 
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Self-Check Responses — 
(J8AL 1 

1. Job and task analyses 

2 S • Review and author task statements and test items 

• Provide resources on existing task analyses and 
measures, industry needs ' 

• Increase, access to employers for verifying J^sk statements 

• Provide detailed information on specific knowledge,^ 
occupatiorial skills, abilities . 

3. Job - a group of related- duties and tasks done by 

or assigned to -one employee 

Duty - collection of tasks describing a group of . 
closely related actions 

Task - distinct measurable action with a specific 
purpose 

4. Action verb,. object, qualifying phrase 

5. Interviews 7 , questionnaires, observation, document review 
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GOAL 2 

1. Information on task importance and frequency of performance 

2. Information on: 

• critical factors in task performance 

• task difficulty 
criteria for judging adequacy of performance 
desirable work habits 

handicapped persons that have performed the job 
successfully 

3. • Have the task list reviewed by your advisory panel. 

• Carefully review available tests to determine if they 
tap' all or some of the competencies identified in 

" your task verification. 

i» 

4. Disadvantaged, handicapped, limited- English proficiency, 
women, and displaced homemakers 
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